Creation of reminders using activity state of an application

ABSTRACT

Systems, methods, and apparatuses are provided for creating and providing reminders that involve an activity states of an application on a device. An activity state can correspond to a particular view within an application, e.g., a view that was reached after several inputs from a user. A user can provide a specification of an activity state in a variety of ways, e.g., providing a reminder command while the application is executing in the foreground with the desired activity state or by voice commands that specify the activity state. A user can provide one or more trigger criteria. Once the reminder is triggered, the specified activity state can be provided to the user.

CROSS REFERENCE OF RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 14/871,115 filed Sep. 30, 2015, which claims the benefit of andpriority to U.S. Provisional Application No. 62/169,351, filed Jun. 1,2015, the entire contents of which are incorporated herein by referencefor all purposes.

BACKGROUND

A user can set a reminder on a device by opening a specific reminderapplication, e.g., a calendar application, and typing text. Such areminder will provide a notification with the user-provided text when aspecified time is reached. Such reminders are certainly useful, but arelimited to the reminder application providing the user's text at acertain time.

Users can also set reminders on mobile devices to provide a notificationwhen the mobile device is close to a particular destination. Forexample, a user can set a reminder to pick up an item when the user isclose to a particular store. In a reminder application on the mobiledevice, the user can write a note that reminds the user of needing theitem, where the note is provided to the user as the notification.However, notes have limited functionality and use.

BRIEF SUMMARY

Embodiments are directed to systems, methods, and apparatuses forcreating and providing a reminder that involves an activity state of anapplication on a device. An activity state can correspond to aparticular view within an application, e.g., a display window that wasreached after several inputs from a user. A user can provide aspecification of an activity state in a variety of ways, e.g., providinga reminder command while the application is executing in the foregroundwith the desired activity state. As another example, a user can providevoice commands that specify the activity state. A user can furtherprovide one or more trigger criteria. Once the reminder is triggered,the specified activity state can be provided to the user.

Other embodiments are directed to systems, portable consumer devices,and computer readable media associated with methods described herein.

A better understanding of the nature and advantages of embodiments ofthe present invention may be gained with reference to the followingdetailed description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart illustrating a method for providing a reminder toa user of a computing device according to embodiments of the presentinvention.

FIG. 2 shows a block diagram of a user interacting with a device tocreate a reminder and the providing of a desired activity state for thereminder according to embodiments of the present invention.

FIGS. 3A and 3B are diagrams illustrating generation of a reminder froma share object of a given display of an executing map applicationaccording to embodiments.

FIG. 4 is a flowchart illustrating a method for creating a reminderusing a current activity state of an executing application according toembodiments of the present invention.

FIG. 5 shows a block diagram of a user interacting with a device tocreate a reminder using a current state of an executing applicationaccording to embodiments of the present invention.

FIG. 6 shows a block diagram of a user interacting with a device tocreate a reminder using voice commands and properties from anapplication according to embodiments of the present invention.

FIG. 7 shows a block diagram of a user interacting with a device tocreate a reminder using voice commands and properties from a mappingindex according to embodiments of the present invention.

FIG. 8 is a block diagram of an example device.

DETAILED DESCRIPTION

Embodiments are directed to systems, methods, and apparatuses forcreating and providing a reminder that involves an activity state of anapplication on a device. The use of an activity state can allow atriggering of a reminder to launch an application with a particularview, such as having a particular document opened, potentially even to aparticular part of the document.

Some embodiments can allow a user to establish a reminder from anapplication that the user is currently using in the foreground. Forexample, a user could establish a reminder that provides a particulardocument, website, or functionality to a user when the reminder istriggered. Triggers for the reminders can include, for example, ageographical location, time, and other criteria. Such reminders can begenerated when a user is at the particular part of an application (e.g.,viewing a particular document, at a listing in an application thatprovides reviews, or at a particular website). For example, a shareobject (or a more specific reminder object) may be displayed on a screenfor the current activity state, and the object may be activated to starta process for creating the reminder using the current activity state. Inother examples, a user can specific an activity state of an applicationusing voice commands, and the specified activity state need not becurrently active at the time of creating the reminder.

An activity state may be defined to bring up a particular view within aparticular application. This may be done by defining the activity stateusing one or more properties, e.g., that are identifiable by theapplication. In this manner, the application can parse the activitydefinition to determine the desired activity state. Such an activitystate definition can encapsulate the state of a user activity in anapplication in a way that allows the same activity to be recalled when areminder is triggered. Activity states can include actions such asediting a document, viewing a web page, or watching a video.

I. Reminders with Activity States

An activity state of an application can be used to make a reminder thatis more convenient. In this manner, the reminder not only reminds theuser of a task (which could be done with just text identifying anapplication), but can also launch the application in the specific statethat the user needs to perform the task. Thus, the user does not need toopen the application, navigate to particular functionality (e.g., to aparticular e-mail, document, or response state). The device canautomatically open the application at the desired screen (example of anactivity state), and the user can complete the task more efficiently.

A. Reminders

A reminder can allow a user to specify an action to be performed on acomputing device (e.g., a phone, tablet, laptop computer, or desktopcomputer) when one or more trigger criteria are satisfied. Examples oftrigger criteria can include a specified time, a specified geographicallocation, and a particular operation on a computing device. When atrigger criterion includes a geographical location, the triggercriterion can be satisfied when the computing device is within aspecified distance of the geographical location.

An example of an action can be providing a particular notification to auser. Typically, a user would open a reminder application and type asubject and then any notes of the reminder, either of which could beprovided in the notification. The notification can serve to remind theuser of a task that needed to be done. The notification can be providedto the user at a specified time according to the one or more triggercriteria, which may be at a time that is convenient for the user toperform the task. But, such notes are limited.

An action can be to launch an application. For example, a musicapplication can automatically launch when a device connects with a car.Applications can also be launched at specific times, e.g., as backgroundprocesses to be performed on a computer. Or, such background processescould be run in response to other triggers, e.g., in response to aconnection or specific communication with another device. But, suchlaunches of applications are not reminders and do not serve a reminderfunction.

A reminder application can display a list of reminders at a particularscreen within the reminder application. Each reminder in the list can bereferred to as a reminder entry. The reminder application can storethese reminder entries and monitor the trigger criteria to determinewhen to perform any action associated with the reminder. The list canshow the subject of the reminders, and selecting a reminder can provideany notes, which may be provided to the user when the reminder istriggered. In embodiments of the present invention, a reminder entry caninclude an activity state of an application, such that the action is tolaunch an application in the particular activity state as a reminder,e.g., for the user to perform a particular task.

B. Activity States

An activity state can correspond to a particular view that is currentlyor can be presented to a user. For example, the state can correspond toa particular navigation screen that a user has reached after selectingone or more options with the application. A mail application can have areply e-mail open for replying to a particular e-mail, and such would bea particular activity state of the mail application. Such an activitystate can be defined by one or more properties, e.g., the identificationof the e-mail to reply and a reply action. Thus, the one or moreproperties can specify an item and an action to be performed on theitem. Further properties can specify a particular location in an item,e.g., a particular location within a file.

Other properties can include the identification of one or more windowsthat are open for an application, e.g., on computing devices that canshow more than one window at a time. As other examples, the propertiescan be a sequence of buttons (or other user interface elements) that auser has selected to reach a particular activity state.

The properties can provide an activity definition, where the activitydefinition is operable to allow the application to provide the activitystate to a user. The activity definition may include further informationthat an application may require for another process to invoke theapplication. For example, the activity definition may include specificapplication protocol interface (API) commands. An API command could bespecific to a particular application, where the one or more propertiesare used to generate the API command.

C. Method

FIG. 1 is a flowchart illustrating a method 100 for providing a reminderto a user of a computing device according to embodiments of the presentinvention. Method 100 can be performed by a computing device, such as aphone, tablet, laptop computer, or a desktop computer. In variousembodiments, method 100 can be performed using voice commands, atouchscreen, keyboard, or a pointing device.

At block 110, request commands can be received from the user to create afirst reminder entry. The request commands can be provided in a varietyof ways, and different request commands can be provided in differentways. For example, one request command can be provided via atouchscreen, and another request command can be provided as a voicecommand.

The request commands can include a reminder command for invoking areminder application to execute on the computing device. The reminderapplication can be any routine(s) that can be used to create a reminder.The request commands can further include one or more trigger criteriafor providing the reminder to the user. As mentioned above, examples oftrigger criteria are a particulate date/time and a particular location.

The request commands can also provide a specification of a firstactivity state of a first application to execute on the computing deviceas part of the reminder. The specification can be provided in a varietyof ways. For example, providing the reminder command while the firstapplication is in the first activity state can function to provide thespecification. As another example, a user can provide one or more voicecommands that, e.g., include one or more properties of the firstactivity state, include tag information (e.g., activity tags) foraccessing a mapping index that maps the tag information to one or moreproperties of the first activity state, or can be used to activate UIelements such that the first application enters the first activitystate, from which the one or more properties can be obtained. Thereminder command, a trigger criterion, and the specification of thefirst activity state can each be provided as a separate request command.

At block 120, the specification can be used to obtain a first set of oneor more properties corresponding to the first activity state. Example ofproperties include: a name of a current screen in the application, anode of the screen in a hierarchy, any terms input by a user, a list ofobjects selected by the user to reach the current state, and the like.The first set of properties may be obtained from a variety of places. Asmentioned above, the first set of properties may be obtained from theuser in the request commands. The first set of properties can beobtained by querying the first application, e.g., when the firstapplication is currently providing the first activity state. The firstset of properties can be obtained by using the specification from theuser to query a mapping index, e.g., that maps voice commands to the oneor more properties of an activity state.

At block 130, a first activity definition corresponding to the firstactivity state is created using the first set of properties. The firstactivity definition can be operable to allow the first application toprovide the first activity state to a user. The first activitydefinition can used by the reminder application or a system routine toinvoke the first application to launch in the first activity state.

In some embodiments, the first activity definition can include a URL,e.g., corresponding to a particular page of a website, potentiallyincluding any user provided information as arguments. Thus, the firstapplication may be a network browsing application. The request commandscan be provided while the user is using a second application, which maycorrespond to the website. The second application can provide the URL inresponse to a request of a current state from the reminder application.The providing of the URL may be done, if the second application does notallow the functionality of being invoked directly, or potentially theURL may be provided in addition (e.g., in case the reminder is launchedon a different device that does not include the second application).Then, the URL can be used to launch the network browsing applicationwhen the reminder is triggered.

At block 140, the first reminder entry is generated. The first reminderentry can include the first activity definition and the one or moretrigger criteria. The reminder application can monitor sensors (e.g., alocation sensor or a clock) to determine when the one or more triggercriteria are satisfied. For example, the reminder application can checkwhether any reminder entries are triggered for a given time.

The first reminder entry can be provided in a list of reminders, whichmay be accessed in the reminder application. When a user accesses thefirst reminder entry, the first activity definition can be provided tothe user, e.g., as a link such that the first activity state can beprovided to the user at that time. The text of the link may be a shortedform of the activity definition, e.g., a form that can be readable by aperson.

At block 150, the computing device can detecting that the one or moretrigger criteria are satisfied. The detection can involve variousroutines and components of the computing device. For example, thecomputing device can monitor one or more sensors of the computingdevice. Such sensors can include a clock, a location sensor (e.g., GPS),and motion sensors, such as an accelerometer, a gyrometer, a compass orother sensors.

At block 160, the first application can be launched responsive to thedetection that the one or more trigger criteria are satisfied. The firstapplication can be launched immediately after detecting the one or moretrigger criteria are satisfied, or launched at a later time. Forexample, a user can be provide an option to launch, e.g., by selectingan OK or confirm UI element. Thus, a user can be provided with anotification of whether the user wishes to proceed with launching thefirst activity state. The first application can be launched furtherresponsive to an affirmative user response. Such launching is stillresponsive to the detection, but not necessarily immediate.

At block 170, the first activity definition in the first reminder entryis used to provide the first activity state of the first application tothe user. For example, the first activity definition can be provided toa system routine (e.g., a launcher application) that launches the firstapplication and provides all or some of the first activity definition tothe first application. As another example, a system routine can provideparts of the first activity definition in a particular sequence, witheach part causing the first application to move to a new state, with thefinal state corresponding to the first activity state.

A similar procedure can be repeated for a second application. Forexample, a second reminder entry can include a second activitydefinition corresponding to a second activity state of a secondapplication. A second request can be received from the user, and thesecond reminder entry can be created via any of ways described herein.

D. System

FIG. 2 shows a block diagram 200 of a user 210 interacting with a device220 to create a reminder and the providing of a desired activity statefor the reminder according to embodiments of the present invention. InFIG. 2, device 220 includes a first application 230 that has a firstactivity state 235. In some embodiments, first application 230 canprovide first activity state 235 to user 210 in response to the user 210interacting with first application 230. For example, a user might selecta particular view of a map application or open a particular e-mail of amail application, both examples of a first application. Firstapplication 230 would have many other activity states (not shown). Firstactivity state 235 can be provided to user 210 before the creation ofany reminder, e.g., to allow user 210 to specify first activity state235 for the reminder.

Reminder application 240 can create reminders based on requests fromuser 210. After creation of a reminder, reminder application 240 canstore the reminders as reminder entries 244. The reminder entry caninclude any text received from the user (e.g., a title of the reminderand any notes to be provided in a notification), one or more triggercriteria, and a specification of the first activity state. Reminderapplication 240 can include a trigger module 242 for determining whenthe one or more trigger criteria are satisfied. As an example, sensors250 can provide information (e.g., time, location, etc.) to triggermodule 242 to determine when the one or more trigger criteria aresatisfied. Trigger module 242 can send a command to reminder applicationor a system routine (e.g., a launcher application) to launch firstactivity state 235 of first application 230 as part of providing thereminder to user 210.

At 201, user 210 sends a reminder request to device 220. The reminderrequest can include one or more request commands (e.g., as received inblock 110 of FIG. 1). Reminder request can be sent at various times,e.g., when first activity state 235 is displayed to a user. As anotherexample, reminder request 201 can be received as voice commands thatspecify first activity state 235.

At 202, reminder application 240 can create a first activity definitionthat corresponds to first activity state 235. The first activitydefinition can be stored in a first reminder entry in reminder entries244 (e.g., in a table). The definition may be formed from one or moreproperties of first activity state 235. The one or more properties canbe determined in various ways. For example, voice commands of reminderrequest 201 can include the one or more properties in a specification ofthe first activity state 235. Reminder application 240 could query firstapplication 230 to obtain the one or more properties, e.g., when firstactivity state 235 is being displayed to user 210. Further, reminderapplication 240 could query a mapping index in device 220 or on anotherdevice (e.g., a server) using the specification received from the user.The index can store information to create definitions based onparticular specifications. For example, the index can store a mappingbetween voice commands and a particular activity definition.

At 203, in response to the one or more trigger criteria being satisfied,reminder application 240 can send a launch command to first application230 to launch first activity state 235. The launch command can be sentdirectly or indirectly to first application 230, e.g., reminderapplication 240 can provide the activity definition to a launchingapplication that can launch first application 230. For example, triggermodule 242 can determine that the one or more trigger criteria aresatisfied. The first reminder entry can be retrieved, and the firstactivity definition can be used to send the launch command to firstapplication 230.

At 204, first activity state 235 can be sent in a reminder display touser 210. Such a display can be a screen showing first activity state235, potentially with a reminder note (e.g., “send reply e-mail” or“plan waking tour of San Francisco”).

The activity definition can include URL that corresponds to firstactivity state 235. First application 230 can define a specific URL sothat a corresponding part of a website can be accessed. In this manner,functionality can be obtained in situations where first application 230does not have an API for launching a particular state. A URL can also beuseful when the reminder is viewed on a different device, and the otherdevice does not have first application 230 installed, but a provider offirst application 230 maintains a website with similar functionality.The URL of the activity definition would correspond to the desiredfunctionality of the website.

Some client applications can allow the reminder application to retrievetitle information from a current activity state. Thus, if a clientapplication opts in, the application can provide convenient titles andany other items that can show up in these reminders. Further, theapplication can use currently selected text to define an activity state,and such selected text can show up in a reminder (e.g., as a note), orbe highlighted when the activity state launched as part of a reminder.

II. Creating Reminders from within a Specific Application

A user can be in the middle of an activity when the user desires tocreate a reminder. For example, the user can be browsing a website orchecking their e-mail, which prompts them to create a reminder.Therefore, exemplary embodiments allow a user to create a reminderaccording to the user activity.

A. Touch Screen Example

A reminder can be created by using a share function of the device. Forexample, if a user is browsing a webpage on their internet browser andwould like to create a reminder based on the webpage, the user canselect a share function of their internet browser to generate areminder. Other examples include a user interacting with a mapapplication or a messaging application. In some embodiments, a shareobject can be used to access a share menu that has an option ofselecting to create a reminder using the current state of theapplication. Generation of a reminder using a share menu is described inmore detail with respect to FIGS. 3A and 3B. In other embodiments, areminder object with just reminder functionality can be used instead ofa share function.

FIGS. 3A and 3B are diagrams illustrating generation of a reminder froma share object of a given display of an executing map applicationaccording to embodiments. FIG. 3A shows a display screen 300 of a userdevice. The device is currently displaying a map view 305 of a mapapplication. When a user desires to create a reminder based on thelocation currently being viewed by the user, the user can select a shareicon 310 (or other share object or a reminder object). Although shareicon 310 may be used to provide multiple options for sharing an activitystate, some embodiments may use a reminder object (a specific type ofshare object) on a screen provided with the current activity state,where the reminder object is used to only create a reminder. An icon isan example of a user interface element, of which other examples arebuttons, tabs, and the like.

As shown in FIG. 3B, a share menu 320 appears after selecting share icon310 and provides the user with an option to create a reminder. The usercan create a reminder from share menu 320 by selecting a reminder icon325. A reminder can then be generated that corresponds to the locationbeing viewed by the user, potentially with any current settings in theviews, e.g., any pins. Other options are available, e.g., to send a linkof the map activity state to another person, e.g., via message or mail.Therefore, a reminder can be created while a user is viewing a screen ofan application. Selecting reminder icon 325 can activate a reminderapplication for generating the reminder.

Once the reminder application is activated for the map application, auser can provide one or more trigger criteria, e.g., date/time orlocation. For example, a user might want to be reminded of a particularlocation of a new city that the user is going to visit, where thereminder can be provided when the user arrives in the city. A user canalso provide text, such as notes and a title. Such information can beprovided via any input device, e.g., a touchscreen or a pointing device(e.g., a mouse, along with a keyboard, or by voice commands).

As other example, a reminder can be created for an e-mail application.For instance, a user can review an e-mail, for example, on the user'slaptop, when the used desires to create a reminder. A reminder can begenerated using an activity state of the e-mail application that iscurrently being viewed by the user. Once the reminder entry is created,the user can view the reminder entry in a list, where the reminder entrycan be displayed with an e-mail application icon to indicate to the userthat the reminder is directed to an e-mail application. Such icons canbe used for other applications. The reminder entry can include a link tothe activity state definition, thereby allowing a user to activate thatstate at some time before the trigger occurs by selecting the link.

As another example, a reminder can be generated while browsing awebpage. A user can be browsing a website on their Internet browser whenthe user desires to create a reminder. A reminder can be generatedaccording to the webpage being used by the user. For example, a remindercan be created to include a URL to the webpage that was being viewed bythe user. Therefore, when the user views the reminder entry, a URL tothe corresponding webpage can be shown in the reminder entry. Althoughan e-mail application and an Internet browser are described above, theseare merely examples, and various types of activity states can be used tocreate a reminder. For example, various types of links can be used todirect the user to an application or website.

B. Method

FIG. 4 is a flowchart illustrating a method 400 for creating a reminderusing a current activity state of an executing application according toembodiments of the present invention. As with other methods describedherein, method 400 can be performed by a computing device.

At block 410, a first activity state of a first application is providedto a user as part of the user using the first application on thecomputing device. The first activity state can correspond to any viewthat a user reaches after interacting with the first application. Forexample, a user might have selected various UI elements to reach aparticular document or other type of file, information about aparticular business (e.g., in a reviewer application), particularcommunications with a particular person in a messaging application, andso on.

At block 420, a first request to create a first reminder entry for thefirst activity state is received while the user is in the first activitystate. For example, a share object on a display screen can be selectedby a user. The first request can invoke a reminder application toexecute on the computing device. The reminder application can be used tocreate a first reminder entry corresponding to the first activity state.

At block 430, a query can be sent to the first application to obtain afirst set of properties while the first application is in the firstactivity state. The first application can receive the query anddetermine the current activity state, which may be tracked by the firstapplication. The first application can check any settings or othervalues used to provide the particular view and determine whether theuser has added any information to the activity state (e.g., any text).

In some embodiments, the first application can provide a title, otherinformation, and one or more properties of the current activity statebecause the first application knows the current state provided to theuser. If the first application does not support internal activitystates, the first application can provide a URL instead, which can beused by a network browsing application to reach functionality on awebsite that corresponds to the first activity state. For example, anapplication can provide an entry ID for a particular business beingviewed, where the entry ID is included in the URL according to a formatthat the corresponding website can understand.

At block 440, the first set of properties can be used to create a firstactivity definition corresponding to the first activity state. The firstactivity definition can be operable to allow the first application toprovide the first activity state to a user. Block 440 can be implementedin a similar manner as block 130 of FIG. 1.

At block 450, one or more trigger criteria are received for providingthe reminder to the user. A examples, the one or more trigger criteriacan be received by a graphical user interface (GUI) or via voicecommands. Examples of trigger criteria are described herein.

At block 460, the first reminder entry is created and can include thefirst activity definition and the one or more trigger criteria. Block460 can be implemented in a similar manner as block 140 of FIG. 1.

At block 470, the computing device can detecting that the one or moretrigger criteria are satisfied. Block 470 can be implemented in asimilar manner as block 150 of FIG. 1.

At block 480, responsive to the detection, the first application can belaunched to provide the first activity state to the user. Block 470 canbe implemented in a similar manner as blocks 160 and 170 of FIG. 1. Forexample, the first activity definition in the first reminder entry canbe used to launch the first activity state.

C. System for Creation of Activity State and Triggering Reminder

FIG. 5 shows a block diagram 500 of a user 510 interacting with a device520 to create a reminder using a current state of an executingapplication according to embodiments of the present invention. In FIG.5, device 520 includes a first application 530 that has a first activitystate 535. In some embodiments, first application 530 can provide firstactivity state 535 to user 510 in response to the user 510 interactingwith first application 530. For example, a user might select aparticular e-mail of an e-mail application. First application 530 wouldhave many other activity states (not shown), e.g., for other e-mailsopened and other actions for an e-mail, e.g., forward instead of reply.

A reminder application 540 can create reminders based on requests fromuser 510, store the reminders as reminder entries 544. Reminderapplication 540 can include a trigger module 542 for determining whenthe one or more trigger criteria are satisfied. As an example, sensors550 can provide information (e.g., time, location, etc.) to triggermodule 542 to determine when the one or more trigger criteria aresatisfied. Trigger module 542 can send a command to launch firstactivity state 535 of first application 530 as part of providing thereminder to user 510.

At 501, first activity state 535 can be provided to user 510 before thecreation of any reminder, e.g., to allow user 510 to specify firstactivity state 535 for the reminder. The initial display indicates thata later display can occur when the reminder is triggered. The initialdisplay of the first activity state can include an object for the userto select to invoke a reminder application to create a reminder.

At 502, user 510 sends a reminder request to device 520. The reminderrequest can be sent while first activity state 535 is provided to user510. For example, a user can select a share icon (e.g., 310 of FIG. 3)to provide the reminder request. First application 530 can be acurrently executing application that is providing information to adisplay of the device. First activity state 535 can be the front-mostactivity being provided to a user.

At 503, reminder application 540 can send a query to first application530 about the current state of the application. First application 530can determine the current activity state as first activity state 535.First application 530 can determine one or more properties for firstactivity state 535. For example, any state variables that have beenspecified to result in first activity state 535 can be obtained. Anapplication may be configured to track such state variables, orinternally query certain routines to obtain such properties.

At 504, first application 530 can send the one or more properties toreminder application. Communications between first application 530 andreminder application 540 can occur via function calls that specify anapplication identifier and that are managed by system routines. Forexample, operating system routines can route the function callsappropriately.

At 505, reminder application 540 can create a first activity definitionthat corresponds to first activity state 535 and store the firstactivity definition in a first reminder entry in reminder entries 544(e.g., in a table). The first activity definition can be formed from oneor more properties of first activity state 535.

At 506, in response to the one or more trigger criteria being satisfied,reminder application 540 can send a launch command to first application530 to launch first activity state 535. For example, trigger module 542can determine that the one or more trigger criteria are satisfied. Thefirst reminder entry can be retrieved, and the first activity definitioncan be used to send the launch command to first application 530.

At 507, first activity state 535 can be sent in a reminder display touser 510. Such a display can be a screen showing first activity state535, potentially with a reminder note.

III. Voice Interface

In some embodiments, one or more of the request commands can be voicecommands. For example, a user can provide a reminder command in the formof “remind me to . . . .” The user can further provide one or moretrigger criteria by voice commands. For example, a user can say “at 3:00PM tomorrow” or any other date/time, and/or a user can say “when Iarrive home” or any other location.

Further, a user can verbally provide a specification of a first activitystate of a first application to execute on the computing device as partof the reminder. The first application could be currently executing andproviding a user interface on a display screen of the device, but notnecessarily, e.g., the application may be running in the background ornot running at all. If the first application was currently providing thefirst activity state, the voice command to provide a specification couldbe simpler, as the device already has information about the firstactivity state.

By using voice commands, embodiments can restrict the use of activitystates only to reminders. This privacy restriction to reminders can beeasier than embodiments using a touchscreen interface, particularly whenthe touchscreen interface is utilized for other types of sharing,besides reminders.

A. Foreground

An application is running in the foreground when the application isproviding a user interface, e.g., by displaying a window to a user. Whenapplication is running in the foreground, the user might be able to usethe semantic term “this” in a request command to refer to the currentactivity state. For example, the user can refer to a reminder for “thisemail,” “this document,” “this map view,” and any other current activitystate.

With the current activity state being identified, a user can alsoprovide a particular action besides simply “remind me.” Thus, a usercould request for the current activity state to be provided when one ormore trigger criteria are satisfied, but a user can also specifyadditional information to tailor the activity state specifically to howthe activity state should be provided. For example, a user could specify“remind me to reply to this email,” where “to reply” can furtherindicate that the activity state to be provided in a reminder is notjust the particular email, but a reply window for replying to theparticular email. Thus, a user can add additional properties to thecurrent activity state (e.g., just viewing a particular e-mail) as partof providing the specification for the first activity state to beprovided for the reminder.

As for how to provide the activity state, a user could say “remind meabout this email and highlight the subject line.” When the email isprovided to the user, the subject line can be highlighted as requested.

A user can also specify a title and/or notes, where the notes may beprovided in a notification when the reminder is triggered. For example,a user could say “remind me about this email as I need to discuss withGeorge.” The email can be displayed, thereby providing the propercontext to the user, and a user could be provided a notificationincluding the words “need discuss with George.” Thus, the more data thatis provided, the more the reminder application can configure thereminder for the user.

B. Background

An application does not need to be running in the foreground for a userto specify an activity state of the application. Instead, a user canidentify a particular application via a voice command, and provide aspecification of the activity state. In the foreground example, a usercan specify a reply window for a particular e-mail that is open. In asimilar manner, a user can provide sufficient description to identifythe e-mail, such as “the e-mail from Susan Chase received yesterday onmy work e-mail account.” As another example, one could specify “a mapview of San Francisco,” potentially with a particular zoom setting, suchas “a map view that just San Francisco.” The map application candetermine a zoom level that provides just the outer boundary of SanFrancisco.

In some implementations, a user can prompt a device to begin a voicecommand interface. The user can enter a reminder mode with keywords,e.g., with “remind me,” “reminder,” and the like. A voice commandinterface may require activation, e.g., by a certain pattern of pushinga button or a certain gesture of moving the device. The informationabout the application and any actions to get to the desired activitystate can be stored for when the reminder is triggered, so as to launchthe application as the desired activity state.

Another example of information for a specification of an activity statecan include: find important e-mails (e.g., ones that are flagged), whichcan result in one reminder for many e-mails or multiple reminders foreach of the important e-mails. The user can be prompted for furtherclarification, if needed, before creating the reminder. For example, ifa user just provided a first name Dave, the device can ask which Dave ismeant. If multiple e-mails or other communications satisfy thespecification, then the device can request further information beprovided for the specification.

C. System where Properties Determined from Application

FIG. 6 shows a block diagram 600 of a user 610 interacting with a device620 to create a reminder using voice commands and properties from anapplication according to embodiments of the present invention. In FIG.6, device 620 includes a first application 630 that has a first activitystate 635. Aspects of other devices, e.g., of FIGS. 2 and 5 can beimplemented for FIG. 6. Although described with respect to voicecommands, aspects can be used for implementations not using voicecommands.

A reminder application 640 can create reminders based on requests fromuser 610, store the reminders as reminder entries 644, and include atrigger module 642 for determining when the one or more trigger criteriaare satisfied. As an example, sensors 650 can provide information (e.g.,time, location, etc.) to trigger module 642 to determine when the one ormore trigger criteria are satisfied. Trigger module 642 can send acommand to launch first activity state 635 of first application 630 aspart of providing the reminder to user 610.

At 601, user 610 sends a reminder request as a voice command to device620. For example, user 610 can start a voice command mode, e.g., with aparticular movement of device 620 or activation of a button on device620. The user can then provide one or more keywords, such as “remind me”or “reminder.” First application 630 could be a currently executingapplication that is providing information to a display of device 620 ata time of receiving reminder request 601, with first activity state 635being the front-most activity. In other embodiments, the firstapplication 630 may not be currently executing on device 620.

In some embodiments, the reminder request can include the specificationcorresponding to first activity state 635. For example, a user canspecify actions the user would take in order to reach first activitystate 635 with first application 630 executing in the foreground. Suchuser actions can be used to query first application 630 or to manipulatefirst application 630 to reach first activity state 635, where firstapplication 630 could then be queried for its current activity state.

At 602, reminder application 640 can send a query to first application630. Reminder application 640 can use the specification received fromuser 610 to formulate the query. The query may be needed when thespecification does not contain the precise commands to launch firstactivity state 635 directly, although the specification can identifyfirst activity state 635. Such a query process can be used when thespecific launch commands are desired for storing in reminder entries644. In other embodiments, the specification from the user may includeone or more properties for determining an activity definition forlaunching first activity state 635, or the one or more properties may beequivalent to the words of the request commands.

At 603, first application 630 can determine one or more propertiescorresponding to first activity state 635 and send the one or moreproperties to reminder application 640. As part of the query, a routineon device 620 can provide commands corresponding to the specificationprovided by the user, where the commands can update the state of firstapplication 630. For example, the voice interface can simulateactivation of buttons on a screen of first application 630 to reachfirst activity state 635. First application 630 can then determine acurrent activity state that results from the commands, which wouldcorrespond to first activity state 635.

At 604, reminder application 640 can create a first activity definitionthat corresponds to first activity state 635 and store first activitydefinition in a first reminder entry in reminder entries 644 (e.g., in atable). The definition is formed from one or more properties of firstactivity state 635.

At 605, in response to the one or more trigger criteria being satisfied,reminder application 640 can send a launch command to first application630 to launch first activity state 635. For example, trigger module 642can determine that the one or more trigger criteria are satisfied. Thefirst reminder entry can be retrieved, and the first activity definitioncan be used to send the launch command to first application 630.

At 606, first activity state 635 can be sent in a reminder display touser 610. Such a display can be a screen showing first activity state635, potentially with a reminder note.

D. System where Properties Determined from Index

As mentioned above, the specification of an activity state by a user canbe equivalent to the one or properties of an activity state and to anactivity definition. But, the voice commands for specifying an activitystate may differ from the one or more properties and the activitydefinition. A user may not typically speak in the same manner that theproperties and definition need to be specified internally on a device.In such a situation, the specification may need to be translated. Such atranslation can be performed at least partially by the application whoseactivity state is being determined. However, such a translationrequirement of a client application can be burdensome on a developer.Accordingly, in some embodiments, a mapping index can be created thattranslates a specification to the one or properties for creating anactivity definition.

The operating system of the device and/or a server in communication withthe device can know all the possible activity states of each of theapplications. For example, the developer can be required to providearchitecture and/or API information before an application can be madeavailable on a site from which an application can be installed.Accordingly, a plurality of applications at a store website can beindexed so that the translation (mapping) from voice commands toactivity states can be determined so as to identify properties forcreating the activity definition.

FIG. 7 shows a block diagram 700 of a user 710 interacting with a device720 to create a reminder using voice commands and properties from amapping index according to embodiments of the present invention. In FIG.7, device 720 includes a first application 730 that has a first activitystate 735. Aspects of other devices, e.g., of FIGS. 2, 5, and 6 can beimplemented for FIG. 7. Although described with respect to voicecommands, aspects can be used for implementations not using voicecommands.

A reminder application 740 can create reminders based on requests fromuser 710, store the reminders as reminder entries 744, and include atrigger module 742 for determining when the one or more trigger criteriaare satisfied. As an example, sensors 750 can provide information (e.g.,time, location, etc.) to trigger module 742 to determine when the one ormore trigger criteria are satisfied. Trigger module 742 can send acommand to launch first activity state 735 of first application 730 aspart of providing the reminder to user 710.

At 701, user 710 sends a reminder request as a voice command to device720. The reminder request can include the specification corresponding tofirst activity state 735. As an example, the user can specify activitytags that correspond to first activity state 635, e.g., for use with amapping index. Such a specification can be used to obtain one or moreproperties of a particular activity state for creating an activitystate.

At 702, reminder application 740 can send a query to a mapping index760. Reminder application 740 can use the specification received fromuser 710 to formulate the query. For example, activity tags in the userspecification can be used to generate the query. The query may be neededwhen the specification does not contain the precise commands to launchfirst activity state 735 directly, although the specification canidentify first activity state 735. Such a query process can be used whenthe specific launch commands are desired for storing in reminder entries644.

At 703, mapping index 760 can map the specification (e.g., certainkeywords/“activity tags” in the voice commands) to one or moreproperties corresponding to first activity state 735 and send the one ormore properties to reminder application 740. Mapping index 760 could beon device 720 or on a server in communication with device 720. Mappingindex 760 can be generated by a server computer connected with a sitethat includes applications for installing on devices.

Mapping index 760 can be implemented in a variety of ways, and multipletechniques may be available to one device and be dependent, e.g., on theway the specification is formulated. If the specification providecertain activity tags in a form that can be mapped directly to one ormore properties of the activity state, then a lookup in a table can bemade with the activity states. For example, each activity tag could beused separately to determine a particular property or subset of theproperties. In another embodiment, the activity tags can map to anentire activity state, and the activity state can then be mapped to aset of one or more properties. For example, an activity record caninclude the one or more properties as fields. Thus, such a mapping wouldcollectively use the activity tags. Sometimes a single voice commandcould correspond to multiple properties of an activity state.

At 704, reminder application 740 can create a first activity definitionthat corresponds to first activity state 735 and store first activitydefinition in a first reminder entry in reminder entries 744 (e.g., in atable). The definition can be formed from one or more properties offirst activity state 735.

At 705, in response to the one or more trigger criteria being satisfied,reminder application 740 can send a launch command to first application730 to launch first activity state 735. For example, trigger module 742can determine that the one or more trigger criteria are satisfied. Thefirst reminder entry can be retrieved, and the first activity definitioncan be used to send the launch command to first application 730.

At 706, first activity state 735 can be sent in a reminder display touser 710. Such a display can be a screen showing first activity state735, potentially with a reminder note.

IV. Activity State of an Application

An application can have any number of activity states. The number ofactivity states available for launching can depend how many states thedeveloper of the application has allowed access. Below are some examplesof implementations using activity states. Further details about activitystates can be found in U.S. Provisional Patent Application 62/005,781,entitled “Activity Continuation Between Electronic Devices,” filed May30, 2014; U.S. patent application Ser. No. 14/586,566, entitled“Activity Continuation Between Electronic Devices,” filed Dec. 30, 2014;and U.S. patent application Ser. No. 14/691,440, entitled “ActivityContinuation Between Electronic Devices,” filed Apr. 20, 2015, thedisclosures of which are incorporated by reference in their entirety.

A. Definition and Use of Activity State

An activity state can be defined as a stand-alone collective unit ofinformation that can be used by an application to provide a particularscreen to which a user would otherwise have to navigate. An activitystate definition can be provided without any dependencies on any otherinformation, and thus an application (on a same or different device) canuse the activity state definition to provide the particular navigationscreen without needing to obtain other information.

An activity state class can represent an instance of a user activity.The activity state class can encapsulate the state of the application ina way that can be continued in the application at a later date as partof a reminder or on other devices, which may use a related application,e.g., a browser for accessing a related website.

To create an activity state, the originating application can create anactivity state object and call a function to send the activity stateobject to a reminder application for storing in a reminder entry. Anactivity state object can include a title and state information, e.g.,information that a user has selected or entered to reach a particularactivity state. The state information can be used to pass native datatypes or custom objects to the receiving device. Examples of native datatypes are Array, Data, Date, Dictionary, Null, Number, Set, String,unique user ID (UUID), and URL.

Once an instance of an activity state object becomes current within anapplication, the operating system or reminder application canperiodically invoke an update routine to give a chance to update theactivity state. The original application itself can update activitystate objects, e.g., as a user is interfacing with the application, sothat an activity state is available when a reminder is requested. Forexample, state information can be added by a function that takesarguments and update the state information of the specified application.Another function can be used to delete state information, or a combinedfunction can add and delete.

When the application launches upon activation of the reminder, anapplication delegate can call a continue function and then obtain theactivity state, e.g., as part of a command received from the reminderapplication. The continue function can then retrieve the activity stateto re-create the particular activity state that the user wanted toinclude in the reminder. In another embodiment, the continue functioncan be called with an argument of the activity state, e.g., when theactivity state has already been received from the reminder application.The user can then immediately finish a task associated with thereminder.

B. Activity Type

In some embodiments, when an activity state is created, an activity typefor the activity can be specified. An activity type can be a uniquestring, e.g., in reverse DNS notation. Each application that is capableof receiving an activity state can declare the activity types that itwill accept, e.g., like declaring the URL schemes the applicationsupports. In some embodiments, for non-document based applications,activity types can be listed under an activity types key at the toplevel of an information property list.

For an application to support a given activity, there can be variousrequirements. For example, the application can have an activity typeentry for the activity type. The application can also be required tohave a developer certificate. If a reminder is to be viewed on anotherdevice, both applications can be required to originate from the samedeveloper with the same developer team ID.

C. Receiving the Activities

When the application is launched as part of a reminder, an applicationdelegate can perform at least some of the processing of the incomingactivity state. Assuming that activity state definition is properlyformed and retrieved, the operating system can call a continue functionfor the application. The activity state instance can be analyzed at thispoint.

The activity state can be passed to a root view controller of theapplication's window and return true, which can tell the operatingsystem (OS) that the activity state has been handled successfully. Fromthis point on, the application can forward calls and restore theactivity. To call on the root view controller, a restore activity statefunction can be invoked. The OS can use this method to tell a receiverto restore an instance of the activity state.

In some embodiments, to reach the desired activity state, theapplication can move down a view controller hierarchy and pass theactivity from the parent to child view controllers until the processreaches the point where the activity is consumed (e.g., all of theactivity state information has been used).

D. Use in Reminder

When the one or more trigger criteria are satisfied and once a reminderapplication retrieves the activity definition, the OS can launch thecorresponding application (i.e., corresponding to the activitydefinition). Once the application is launched, the OS can call on acontinue application function. At this point the application may nothave received the activity state instance. The OS can transfer data fromthe reminder application to the original application (which could be onanother device if the reminder acts on multiple devices); such data caninclude any notes or information that might be displayed within theapplication.

It is possible that a new version of the application has been installedin between the creation of the reminder and a trigger of the reminder.In some implementations, a version number can be added to each reminderentry, and activity state definitions may only be accepted with a sameversion number (or potentially earlier).

Some embodiments can avoid using platform specific values like a contentoffset of a scroll view, as it can be better to use relative landmarks.For example, if the user is viewing some items in a table view, anapplication can pass the index path of the top most visible item in thetable view in the activity state object instead of passing the contentoffset or visible part of the table view.

E. URLs

In some embodiments, a URL can be used in a definition for an activitystate. This may be useful when the reminder is viewed on a differentdevice that was used to initially create the reminder, where the otherdevice may not have the application installed. Instead, the reminder canprovide a URL as the activity state definition. The other device canrecognize a URL in the activity state definition and use the URL in abrowser to go to a designated Web page.

In some embodiments, the reminder entry can include both a URL and anapplication-specific activity state (both of which are activity statedefinitions). The other device can use an application ID to determinethat a suitable application is not installed, and then use the URLinstead in combination with launching a browser. For example, a reviewapplication can also be accessed on a website to see a particularbusiness and corresponding reviews, e.g., if the review application isnot installed.

Some applications could provide only URLs, and thus a reminder for suchan application can always launch a browser when the one or more triggercriteria is satisfied. The application can identify information about acurrent state and then determine corresponding tags for a URL to resultin a same or similar location on a website corresponding to the view inthe application.

If the application is a browser, then an application-specific activitystate could be a URL. The reminder entry can include a title of the Webpage. The domain name can be put in the notes, so that the user can seewhat the reminder is about, since a URL may not be easily recognizable.Further state information can include any text added by a user and whereon the website the user was at, e.g., including a scroll location. Thus,when a user looks at a list of reminders, the title and a clean domaincan be seen such that a user would not be confused by a long URL.

Using a URL can add certain privacy protection, as then an applicationdoes not obtain information about user interactions with theapplication. Thus, on activation of a share sheet from a particularactivity state of an application, the application does not obtain a fullapplication-specific activity state but only a URL. Other applicationsmight not have an associated website and thus a URL might not beappropriate, and an activity state might be allowed.

URLs could include documents on a network, e.g., on the Internet. Someembodiments can be limited to certain types of URLs, e.g., ones that useHTTP or HTTPS. In some implementations, file links can be passed as arelative path and re-constructed to obtain a URL.

V. Launching Activity State in Response to Trigger

A reminder can be created with a particular icon (e.g., a share icon)when the application is in the desired activity state. The title can becreated from user input or information obtained from the particularactivity state (e.g., a particular email that is part of the activitystate). In an email example where the trigger is when the user getshome, the mail application can open. As another example, a notificationcan be provided that provides a user an option to launch the applicationin a particular activity state. In various embodiments, the notificationcan include the title of the reminder and any notes.

Other examples of triggers can be the use of a particular application.For example, when a particular application is opened, it might trigger aparticular document to open in that application, and potentially open toa particular part of that document. For example, a user might open apresentation application, triggering a particular presentation to beopened. Further, notes provided by the user for a particular changemight be provided at that time. Such a reminder might be created byvoice commands when the user is not editing the presentation document.Thus, the reminder can be presented to the user at an opportune time, asopposed to sitting in a separate notes file.

As another example, a reminder can be created for an application tocheck for information that is not available yet. For example, a remindermight automatically bring up a sports application for checking scores ofa particular game that is to occur tomorrow. A particular activity statefor checking such scores can be known, e.g., from the user checkingscores of previous games.

In some embodiments, an activity state might not be reachable at thetime the reminder is triggered. For example, the activity state might befor current news stories, current posts, etc. Accordingly, a user maynot be able to reach a particular story in response to a reminderrequest to read a particular story/post. In some implementations, theactivity of a particular story/post might be saved directly to thestory/post, thereby preserving the ability to read the story/post.

Besides or in addition to a notification (e.g., asking the user whetherto launch the activity state of the reminder), a reminder can have othertypes of alerts. For example, an audio alert can also be provided.

VI. Example Activity States

Activity states can vary as much as different applications vary. Forexample, different applications can have different number of propertiesfor defining an activity state. For activity states that involve aparticular document (e.g., email or file), an activity state can includethe selection of text within the document. Further, the selected textcan be provided in the notes of the reminder.

For a map application, an activity state can include any features(properties) that are used to reach the particular map view. Forexample, an activity state can include a location of the center of theview, a zoom level, and an orientation (e.g., which direction is up onthe screen). Further properties could be any search terms entered by auser and any pins a user has dropped or have resulted from a search.

Applications can determine the level of granularity to provide in anactivity state. For example, an application might only have a few levelsin a hierarchical tree that specifies all the properties of activitystates, whereas another application might have five or more levels.Further, applications may or may not allow additional user informationthat shows up on a screen to be classified as a property of the activitystate. Thus, it can be up to an application to determine how to capturethe activity state. Regardless, embodiments can determine theproperties, even with a mapping index, as long as the properties arepublished by the developer of the application.

VII. Handoff

A user may have multiple devices and these devices can interact witheach other. Therefore, an exemplary embodiment can create a reminder tobe shared between different devices. Specifically, a reminder can becreated at a first device, such as a mobile phone, and the reminder canbe triggered when the user operates another device, such as their laptopor a second mobile phone.

As an example, a user can be reviewing e-mails on a mobile device.However, the user would like to reply to an e-mail at a later time fromtheir laptop. The user can create a reminder on the mobile device (e.g.,using techniques described herein). Once created, the reminder can besent to the laptop. A reminder application on the laptop can thendetermine whether the one or more trigger criteria are satisfied.

For example, the user can create a reminder to “Reply to e-mail when Iam on my laptop.” When the user operates the laptop, the reminder can betriggered, and the user can be reminded to reply to the email. Operationof the laptop can be determined based on, for example, a determinationthat the laptop has been connected to a network or a cloud, and thereminder can be subsequently triggered. A mobile phone and laptop aredescribed above, however, various devices can be used to communicate areminder.

Such reminder can also be sent to another person. Such reminders wouldnot work if the activity state involved a particular document on thesender's device. But, a more general reminder could equally launch onthe other person's device, as long as the other person's device has acorresponding application. In the process, a device could also have areminder application that recognizes the URL and launches a networkbrowsing application, e.g., if the other application was not installedon the other person's device and the activity definition included a URL.

VIII. Example Device

FIG. 8 is a block diagram of an example device 800, which may be amobile device. Device 800 generally includes computer-readable medium802, a processing system 804, an Input/Output (I/O) subsystem 806,wireless circuitry 808, and audio circuitry 810 including speaker 850and microphone 852. These components may be coupled by one or morecommunication buses or signal lines 803. Device 800 can be any portableelectronic device, including a handheld computer, a tablet computer, amobile phone, laptop computer, tablet device, media player, personaldigital assistant (PDA), a key fob, a car key, an access card, amulti-function device, a mobile phone, a portable gaming device, a cardisplay unit, or the like, including a combination of two or more ofthese items.

It should be apparent that the architecture shown in FIG. 8 is only oneexample of an architecture for device 800, and that device 800 can havemore or fewer components than shown, or a different configuration ofcomponents. The various components shown in FIG. 8 can be implemented inhardware, software, or a combination of both hardware and software,including one or more signal processing and/or application specificintegrated circuits.

Wireless circuitry 808 is used to send and receive information over awireless link or network to one or more other devices' conventionalcircuitry such as an antenna system, an RF transceiver, one or moreamplifiers, a tuner, one or more oscillators, a digital signalprocessor, a CODEC chipset, memory, etc. Wireless circuitry 808 can usevarious protocols, e.g., as described herein.

Wireless circuitry 808 is coupled to processing system 804 viaperipherals interface 816. Interface 816 can include conventionalcomponents for establishing and maintaining communication betweenperipherals and processing system 804. Voice and data informationreceived by wireless circuitry 808 (e.g., in speech recognition or voicecommand applications) is sent to one or more processors 818 viaperipherals interface 816. One or more processors 818 are configurableto process various data formats for one or more application programs 834stored on medium 802.

Peripherals interface 816 couple the input and output peripherals of thedevice to processor 818 and computer-readable medium 802. One or moreprocessors 818 communicate with computer-readable medium 802 via acontroller 820. Computer-readable medium 802 can be any device or mediumthat can store code and/or data for use by one or more processors 818.Medium 802 can include a memory hierarchy, including cache, main memoryand secondary memory.

Device 800 also includes a power system 842 for powering the varioushardware components. Power system 842 can include a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light emittingdiode (LED)) and any other components typically associated with thegeneration, management and distribution of power in mobile devices.

In some embodiments, device 800 includes a camera 844. In someembodiments, device 800 includes sensors 846. Sensors can includeaccelerometers, compass, gyrometer, pressure sensors, audio sensors,light sensors, barometers, and the like. Sensors 846 can be used tosense location aspects, such as auditory or light signatures of alocation.

In some embodiments, device 800 can include a GPS receiver, sometimesreferred to as a GPS unit 848. A mobile device can use a satellitenavigation system, such as the Global Positioning System (GPS), toobtain position information, timing information, altitude, or othernavigation information. During operation, the GPS unit can receivesignals from GPS satellites orbiting the Earth. The GPS unit analyzesthe signals to make a transit time and distance estimation. The GPS unitcan determine the current position (current location) of the mobiledevice. Based on these estimations, the mobile device can determine alocation fix, altitude, and/or current speed. A location fix can begeographical coordinates such as latitudinal and longitudinalinformation.

One or more processors 818 run various software components stored inmedium 802 to perform various functions for device 800. In someembodiments, the software components include an operating system 822, acommunication module (or set of instructions) 824, a location module (orset of instructions) 826, a reminder module 828 (e.g., corresponding toreminder applications described herein), and other applications (or setof instructions) 834.

Operating system 822 can be any suitable operating system, includingiOS, Mac OS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embeddedoperating system such as VxWorks. The operating system can includevarious procedures, a plurality of instructions, software componentsand/or drivers for controlling and managing general system tasks (e.g.,memory management, storage device control, power management, etc.) andfacilitates communication between various hardware and softwarecomponents.

Communication module 824 facilitates communication with other devicesover one or more external ports 836 or via wireless circuitry 808 andincludes various software components for handling data received fromwireless circuitry 808 and/or external port 836. External port 836(e.g., USB, FireWire, Lightning connector, 60-pin connector, etc.) isadapted for coupling directly to other devices or indirectly over anetwork (e.g., the Internet, wireless LAN, etc.).

Location/motion module 826 can assist in determining the currentposition (e.g., coordinates or other geographic location identifier) andmotion of device 800. Modern positioning systems include satellite basedpositioning systems, such as Global Positioning System (GPS), cellularnetwork positioning based on “cell IDs,” and Wi-Fi positioningtechnology based on a Wi-Fi networks. GPS also relies on the visibilityof multiple satellites to determine a position estimate, which may notbe visible (or have weak signals) indoors or in “urban canyons.” In someembodiments, location/motion module 826 receives data from GPS unit 848and analyzes the signals to determine the current position of the mobiledevice. In some embodiments, location/motion module 826 can determine acurrent location using Wi-Fi or cellular location technology. Forexample, the location of the mobile device can be estimated usingknowledge of nearby cell sites and/or Wi-Fi access points with knowledgealso of their locations. Information identifying the Wi-Fi or cellulartransmitter is received at wireless circuitry 808 and is passed tolocation/motion module 826. In some embodiments, the location modulereceives the one or more transmitter IDs. In some embodiments, asequence of transmitter IDs can be compared with a reference database(e.g., Cell ID database, Wi-Fi reference database) that maps orcorrelates the transmitter IDs to position coordinates of correspondingtransmitters, and computes estimated position coordinates for device 800based on the position coordinates of the corresponding transmitters.Regardless of the specific location technology used, location/motionmodule 826 receives information from which a location fix can bederived, interprets that information, and returns location information,such as geographic coordinates, latitude/longitude, or other locationfix data.

Application suggestion module 828 can include various sub-modules orsystems, e.g., as described above in FIG. 7. Application suggestionmodule 828 can perform all or part of method 400.

The one or more applications 834 on the mobile device can include anyapplications installed on the device 800, including without limitation,a browser, address book, contact list, email, instant messaging, wordprocessing, keyboard emulation, widgets, JAVA-enabled applications,encryption, digital rights management, voice recognition, voicereplication, a music player (which plays back recorded music stored inone or more files, such as MP3 or AAC files), etc.

There may be other modules or sets of instructions (not shown), such asa graphics module, a time module, etc. For example, the graphics modulecan include various conventional software components for rendering,animating and displaying graphical objects (including without limitationtext, web pages, icons, digital images, animations and the like) on adisplay surface. In another example, a timer module can be a softwaretimer. The timer module can also be implemented in hardware. The timemodule can maintain various timers for any number of events.

The I/O subsystem 806 can be coupled to a display system (not shown),which can be a touch-sensitive display. The display displays visualoutput to the user in a GUI. The visual output can include text,graphics, video, and any combination thereof. Some or all of the visualoutput can correspond to user-interface objects. A display can use LED(light emitting diode), LCD (liquid crystal display) technology, or LPD(light emitting polymer display) technology, although other displaytechnologies can be used in other embodiments.

In some embodiments, I/O subsystem 806 can include a display and userinput devices such as a keyboard, mouse, and/or track pad. In someembodiments, I/O subsystem 806 can include a touch-sensitive display. Atouch-sensitive display can also accept input from the user based onhaptic and/or tactile contact. In some embodiments, a touch-sensitivedisplay forms a touch-sensitive surface that accepts user input. Thetouch-sensitive display/surface (along with any associated modulesand/or sets of instructions in medium 802) detects contact (and anymovement or release of the contact) on the touch-sensitive display andconverts the detected contact into interaction with user-interfaceobjects, such as one or more soft keys, that are displayed on the touchscreen when the contact occurs. In some embodiments, a point of contactbetween the touch-sensitive display and the user corresponds to one ormore digits of the user.

The user can make contact with the touch-sensitive display using anysuitable object or appendage, such as a stylus, pen, finger, and soforth. A touch-sensitive display surface can detect contact and anymovement or release thereof using any suitable touch sensitivitytechnologies, including capacitive, resistive, infrared, and surfaceacoustic wave technologies, as well as other proximity sensor arrays orother elements for determining one or more points of contact with thetouch-sensitive display.

Further, the I/O subsystem can be coupled to one or more other physicalcontrol devices (not shown), such as pushbuttons, keys, switches, rockerbuttons, dials, slider switches, sticks, LEDs, etc., for controlling orperforming various functions, such as power control, speaker volumecontrol, ring tone loudness, keyboard input, scrolling, hold, menu,screen lock, clearing and ending communications and the like. In someembodiments, in addition to the touch screen, device 800 can include atouchpad (not shown) for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad can be a touch-sensitive surface that is separatefrom the touch-sensitive display or an extension of the touch-sensitivesurface formed by the touch-sensitive display.

In some embodiments, some or all of the operations described herein canbe performed using an application executing on the user's device.Circuits, logic modules, processors, and/or other components may beconfigured to perform various operations described herein. Those skilledin the art will appreciate that, depending on implementation, suchconfiguration can be accomplished through design, setup,interconnection, and/or programming of the particular components andthat, again depending on implementation, a configured component might ormight not be reconfigurable for a different operation. For example, aprogrammable processor can be configured by providing suitableexecutable code; a dedicated logic circuit can be configured by suitablyconnecting logic gates and other circuit elements; and so on.

Any of the software components or functions described in thisapplication may be implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java, C, C++, C #, Objective-C, Swift, or scripting language such asPerl or Python using, for example, conventional or object-orientedtechniques. The software code may be stored as a plurality ofinstructions or commands on a computer readable medium for storageand/or transmission. A suitable non-transitory computer readable mediumcan include random access memory (RAM), a read only memory (ROM), amagnetic medium such as a hard-drive or a floppy disk, or an opticalmedium such as a compact disk (CD) or DVD (digital versatile disk),flash memory, and the like. The computer readable medium may be anycombination of such storage or transmission devices.

Computer programs incorporating various features of the presentinvention may be encoded on various computer readable storage media;suitable media include magnetic disk or tape, optical storage media suchas compact disk (CD) or DVD (digital versatile disk), flash memory, andthe like. Computer readable storage media encoded with the program codemay be packaged with a compatible device or provided separately fromother devices. In addition program code may be encoded and transmittedvia wired optical, and/or wireless networks conforming to a variety ofprotocols, including the Internet, thereby allowing distribution, e.g.,via Internet download. Any such computer readable medium may reside onor within a single computer product (e.g. a hard drive, a CD, or anentire computer system), and may be present on or within differentcomputer products within a system or network. A computer system mayinclude a monitor, printer, or other suitable display for providing anyof the results mentioned herein to a user.

Although the invention has been described with respect to specificembodiments, it will be appreciated that the invention is intended tocover all modifications and equivalents within the scope of thefollowing claims.

What is claimed is:
 1. A method for creating and providing a reminder toa user of a computing device, the method comprising, at the computingdevice: receiving a request from the user to create the reminder; afterreceiving the request to create the reminder, receiving request commandsfrom the user to create a first reminder entry, the request commandsincluding: a reminder command for invoking an application to execute onthe computing device, wherein the application is a map application; oneor more trigger criteria for providing the reminder to the user; and afirst activity state of the application to execute on the computingdevice as part of the reminder, wherein the first activity statecomprises a plurality of properties input by the user to obtain aselected view, wherein execution of the first activity state when thefirst reminder entry is triggered causes the application to use theplurality of properties in order to launch to the selected view withinthe application, wherein the selected view is a view within theapplication that is designated by the user at a time the user sends therequest command to create the first reminder entry, and wherein theexecution of the first activity state when the first reminder entry istriggered causes the application to use a first portion of a sequence ofthe plurality of properties to change a state of the application to anintermediate application state and causes the application to use asecond portion of the sequence of the plurality of properties to changethe state of the application to the first activity state; creating afirst activity definition corresponding to the first activity state, thefirst activity definition operable to allow the application to providethe first activity state to the user; in response to receiving therequest commands to create the first reminder entry, generating thefirst reminder entry, including the first activity definition and theone or more trigger criteria; detecting that the one or more triggercriteria are satisfied; and responsive to detecting that the one or moretrigger criteria are satisfied, triggering the first reminder entry,wherein triggering the first reminder entry comprises: launching theapplication; and using the first activity definition in the firstreminder entry to open the application to the selected view within theapplication.
 2. The method according to claim 1, wherein the selectedview is selected by the user when creating the first reminder entry. 3.The method according to claim 2, wherein the selected view for the firstactivity state is received from the user by the user providing thereminder command when the application is in the selected view.
 4. Themethod according to claim 1, wherein the selected view within theapplication comprises a view of the application after a plurality ofinputs have been received from the user while in the application.
 5. Themethod according to claim 1 further comprising, at the computing device:receiving text from the user for including with the first reminderentry; and providing the text in a notification in combination with theselected view within the application.
 6. The method according to claim5, wherein the text comprises information for reminding the user of anaction to be performed in the selected view.
 7. The method according toclaim 5, wherein the provided text is highlighted.
 8. The methodaccording to claim 1, wherein the first activity definition comprisesapplication protocol interface (API) commands corresponding to a type ofthe application.
 9. The method according to claim 1, wherein the firstactivity definition comprises a list of objects selected by the user toarrive at the selected view.
 10. The method according to claim 1,wherein the first activity definition includes a URL directing the userto the selected view.
 11. The method according to claim 1, wherein therequest commands are provided by a touch screen or voice command. 12.The method according to claim 1, wherein the computing device is amobile device, wherein the one or more trigger criteria include ageographical location, and wherein the one or more trigger criteria aresatisfied when the computing device is within a specified distance ofthe geographical location.
 13. The method according to claim 1, whereinthe application is automatically launched by the computing device inresponse to detecting that the one or more trigger criteria issatisfied.
 14. The method according to claim 1, wherein the view withinthe application occurs after the user has selected one or more userinterface (UI) elements within the application.
 15. The method accordingto claim 1, wherein the receiving the request from the user to createthe reminder comprises: receiving a selection of a share function fromthe map application in order to create the reminder.
 16. The methodaccording to claim 1, wherein the receiving the request from the user tocreate the reminder comprises receiving a selection on a touchscreen ofthe computing device to create the reminder.
 17. A computer productcomprising a non-transitory computer readable medium storinginstructions that when executed control a computing device to create andproviding a reminder to a user of the computing device, the instructionscomprising: receiving a request from the user to create the reminder;after receiving the request to create the reminder, receiving requestcommands from the user to create a first reminder entry, the requestcommands including: a reminder command for invoking an application toexecute on the computing device, wherein the application is a mapapplication; one or more trigger criteria for providing the reminder tothe user; and a first activity state of the application to execute onthe computing device as part of the reminder, wherein the first activitystate comprises a plurality of properties input by the user to obtain aselected view, wherein execution of the first activity state when thefirst reminder entry is triggered causes the application to use theplurality of properties in order to launch to the selected view withinthe application, wherein the selected view is a view within theapplication that is designated by the user at a time the user sends therequest command to create the first reminder entry, and wherein theexecution of the first activity state when the first reminder entry istriggered causes the application to use a first portion of a sequence ofthe plurality of properties to change a state of the application to anintermediate application state and causes the application to use asecond portion of the sequence of the plurality of properties to changethe state of the application to the first activity state; creating afirst activity definition corresponding to the first activity state, thefirst activity definition operable to allow the application to providethe first activity state to the user; in response to receiving therequest commands to create the first reminder entry, generating thefirst reminder entry, including the first activity definition and theone or more trigger criteria; detecting that the one or more triggercriteria are satisfied; and responsive to detecting that the one or moretrigger criteria are satisfied, triggering the first reminder entry,wherein triggering the first reminder entry comprises: launching theapplication; and using the first activity definition in the firstreminder entry to open the application to the selected view within theapplication.
 18. The computer product according to claim 17, wherein theselected view is selected by the user when creating the first reminderentry.
 19. The computer product according to claim 17, wherein theselected view for the first activity state is received from the user bythe user providing the reminder command when the application is in theselected view.
 20. A computing device comprising: a memory for storingreminders; one or more sensors; one or more processors configured to:receiving a request from a user to create the reminder; after receivingthe request to create the reminder, receive request commands from theuser to create a first reminder entry, the request commands including: areminder command for invoking an application to execute on the computingdevice, wherein the application is a map application; one or moretrigger criteria for providing the reminder to the user; and a firstactivity state of the application to execute on the computing device aspart of the reminder, wherein the first activity state comprises aplurality of properties input by the user to obtain a selected view,wherein execution of the first activity state when the first reminderentry is triggered causes the application to use the plurality ofproperties in order to launch to the selected view within theapplication, wherein the selected view is a view within the applicationthat is designated by the user at a time the user sends the requestcommand to create the first reminder entry, and wherein the execution ofthe first activity state when the first reminder entry is triggeredcauses the application to use a first portion of a sequence of theplurality of properties to change a state of the application to anintermediate application state and causes the application to use asecond portion of the sequence of the plurality of properties to changethe state of the application to the first activity state; create a firstactivity definition corresponding to the first activity state, the firstactivity definition operable to allow the application to provide thefirst activity state to the user; in response to receiving the requestcommands to create the first reminder entry, generate the first reminderentry, including the first activity definition and the one or moretrigger criteria; detect that the one or more trigger criteria aresatisfied; and responsive to detecting that the one or more triggercriteria are satisfied, trigger the first reminder entry, whereintriggering the first reminder entry causes one or more processors to:launch the application; and use the first activity definition in thefirst reminder entry to open the application to the selected view withinthe application.
 21. The computing device according to claim 20, whereinthe selected view is selected by the user when creating the firstreminder entry.
 22. The computing device according to claim 20, whereinthe selected view for the first activity state is received from the userby the user providing the reminder command when the application is inthe selected view.